| Frederick Jelinek | |
|---|---|
| Born | November 18, 1932 Kladno, now Czech Republic |
| Died | September 14, 2010 (aged 77) Baltimore, United States |
| Citizenship | American |
| Fields | Information theory, natural language processing |
| Institutions | Cornell University, IBM Research, Johns Hopkins University |
| Alma mater | Massachusetts Institute of Technology |
| Doctoral advisor | Robert Fano |
| Notable students | Neil Sloane |
| Known for | Advancement of natural language processing techniques |
| Influences | Roman Jakobson |
| Notable awards |
|
Frederick Jelinek (18 November 1932 – 14 September 2010) was a Czech American researcher in information theory, automatic speech recognition, and natural language processing. He was well-known for his oft-quoted quip that "Every time I fire a linguist, the performance of the speech recognizer goes up".[note 1]
Born in Czechoslovakia just before the war, his family managed to emigrate to the United States in the early years of the communist regime. He studied engineering at the Massachusetts Institute of Technology and taught for 10 years at Cornell University before being offered a job at IBM Research. There his team essentially revolutionized approaches to computer speech recognition and machine translation. After IBM, he went to head the Center for Language and Speech Processing at Johns Hopkins University for 17 years, and he was still working on the day of his death. He had been married since 1961 to Czech screenwriter Milena Jelinek.
Contents |
Bedřich Jelínek[6] was born in Kladno a decade before WWII to Vilém and Trude Jelinek.[7] His father was Jewish, but his mother was a Catholic born in Switzerland who converted to Judaism.[8][9] Jelinek senior, a dentist, had planned early for an escape to England, arranging for a passport, visa, and the shipping of his dentistry materials; the couple planned to send their son to an English private school. However, Vilém decided to stay at the last minute, and was eventually sent to the Theresienstadt concentration camp,[10] where he died of disease in 1945.[7][9] The family was forced to move to Prague in 1941, but Frederick, his sister and mother, thanks to the latter's background, escaped the concentration camps.[9]
"It is generally believed that scientific talent reveals itself in early youth. [...] This was certainly not my case. I somehow slid into my scientific profession. My mother wished for me to become a physician, just like my father. [...] I myself wanted to be a lawyer, defender of the unjustly accused. But my career is the result of political circumstances, academic possibilities, and lucky accidents."
After the war, Jelinek successfully entered in the gymnasium despite having missed several years of schooling (as education of Jewish children had been forbidden since 1942). His mother, anxious for her son to get a good education, made great efforts for their emigration,[note 2] particularly as it became clear he would not be allowed to even attempt the graduation examination. His mother hoped for her son to become a physician, but Jelinek dreamed of being a lawyer; he ended up studying engineering in evening classes at the City College of New York. He received stipends from the National Committee for a Free Europe that allowed him to study at the Massachusetts Institute of Technology. About his choice of specialty, he joked: "Fortunately, to electrical engineering there belonged a discipline whose aim was not the construction of physical systems: the theory of information."[10] He obtained his Ph.D. in 1962, with Robert Fano as his adviser: "Not daring to approach Shannon himself, I asked Professor Fano to be my thesis adviser.".[11][12]
In 1957, Jelinek paid an unexpected visit to Prague. He had been in Vienna and, hoping to see his former acquaintances again, applied for a visa. He met with his old friend Miloš Forman, who introduced him to film student Milena Tabolova, whose screenplay had been the basis for the just released movie Easy Life (Snadný život).[13][14] His flight back to the U.S. had a stopover in Munich, during which he called her to propose.[9] Tabolova was considered a dissident, and her movie did not sit well with the authorities.[14] Jelinek asked for help from Jerome Wiesner and Cyrus Eaton, the latter who lobbied Nikita Khrushchev.[13] Following the inauguration of John F. Kennedy, a group of Czech dissidents were allowed to emigrate in January 1961; thanks to the lobbying, the future Milena Jelinek was one of them.[9][13]
After completing his graduate studies, Jelinek, who had developed an interest in linguistics, had plans to work with Charles F. Hockett at Cornell University. Unfortunately for him, these fell through and during the next ten years he continued to devote himself to information theory.[10] Having previously worked at IBM during a sabbatical, he began work there in 1972, at first on leave for Cornell, but permanently from 1974 on; he remained there for over twenty years. Although at first he was to hold a regular research job, upon his arrival he learned that Josef Raviv had just been promoted to head of the newly opened IBM Haifa Research Laboratory, and found himself head of the Continuous Speech Recognition group at the Thomas J. Watson Research Center.[10][12] Despite his team's successes in this area, his work remained little known in his home country, as scientists were not allowed to participate in key conferences.[13]
After the 1989 fall of communism, he helped with establishing scientific relationships, regularly visiting to lecture and helping to convince IBM to establish a computing centre at Charles University.[8][10][15] In 1993 he retired from IBM and went to Johns Hopkins University's Center for Language and Speech Processing, where he was director and Julian Sinclair Smith Professor of Electrical and Computer Engineering.[11][16] He was still working there at the time of his death; Jelinek died of a heart attack at the close of an otherwise normal workday in mid-September 2010.[9][16] He was survived by his wife, daughter and son, sister, stepsister, and three grandchildren.[16]
Information theory was a fashionable scientific approach in the mid '50s.[12] However, pioneer Claude Shannon mused in 1956 that this trendiness was dangerous: "Our fellow scientists in many different fields, attracted by the fanfare and by the new avenues opened to scientific analysis, are using these ideas in their own problems. [...] It will be all too easy for our somewhat artificial prosperity to collapse overnight when it is realized that the use of a few exciting words like information, entropy, redundancy, do not solve all our problems."[17] Indeed over the next decade, a combination of factors would shut down application of information theory to natural language processing (NLP) problems, in particular machine translation. One was the 1957 publication of Noam Chomsky's Syntactic Structures, which stated that "probabilistic models give no insight into the basic problems of syntactic structure".[18] This accorded well with the philosophy of the artificial intelligence research of the time, which promoted rule-based approaches. The other factor was to be the 1966 ALPAC report, which recommended that the government stop funding research in machine translation. ALPAC chairman John Pierce later characterised that field as filled with "mad inventors or untrustworthy engineers". He argued that the underlying linguistic problems must be solved before attempts at NLP could be reasonably made. Combined, these elements essentially halted research in the field.[5][19]
Jelinek had begun to develop an interest in linguistics after the immigration of his wife, who initially enrolled in the linguistics program of the MIT thanks to Roman Jakobson's help. Jelinek often accompanied her to Chomsky's lecture, and even went so far as to discuss the possibility of changing orientation with his adviser. Fano was "really upset", and with the failure of his project with Hockett at Cornell, he did not return to this avenue of research until starting work at IBM.[12] The scope of research at IBM was considerably different from that of most other teams: "While Fred was leading IBM’s effort to solve the general dictation problem during the decade or so following 1972, most other U.S. companies and academic researchers were working on very limited problems [...] or were staying out of the field entirely."[19]
"He was not a pioneer of speech recognition, he was the pioneer of speech recognition."
It was only natural for Jelinek to see speech recognition as an information theory problem: a noisy channel (in this case the acoustic signal)—and yet this was a daring, or even anathema approach to observers.[5][16][19] The concept of perplexity was introduced in their first model,[12] New Raleigh Grammar, itself published (1976) in the "now famous paper in the Proceedings of the IEEE called "Continuous Speech Recognition by Statistical Methods"'.[5] The basic noisy channel approach "reduced the speech recognition problem to one of producing two statistical models."[5] Whereas New Raleigh Grammar was a hidden Markov model, Tangora (their next model) was broader and involved n-grams, specifically trigrams. Even though "it was obvious to everyone that this model was hopelessly impoverished", it would remained unimproved upon until another paper of Jelinek himself presented in 1999 (see under "selected publication").[5] The same trigram approach was applied to phones in single words. Although the identification of parts of speech turned out not to be very useful for speech recognition, tagging methods developed during these projects are now used in various NLP applications.[12]
The incremental research techniques developed at IBM eventually became dominant in the field after DARPA, in the mid-80s, returned to NLP research and imposed that methodology to participating teams, shared common goals, data, and precise evaluation metrics.[19] The Continuous Speech Recognition Group's research, which required large amounts of data to train the algorithms, eventually led to the creation of the Linguistic Data Consortium. In the 80s, although the broader problem of speech recognition remained unsolved, they sought to apply the methods developed to other problems, and came up with two: machine translation and stock value prediction. In fact, a group of IBM searchers eventually went to work for Renaissance Technologies. Jelinek comments: "The performance of the Renaissance fund is legendary, but I have no idea whether any methods we pioneered at IBM have ever been used. My former colleagues will not tell me: theirs is a very hush-hush operation!"[12] Methods very similar to those developed for achieving speech recognition are at the base of most machine translation systems today.
Observers[5][19] have noted that Pierce's paradigm, according to which engineering achievements in this area would be built on scientific progress, has been inverted, with the achievements in engineering being at the base of a number of scientific findings.
Jelinek's works won "best paper" awards on several occasions, and he received a number of company awards while he worked at IBM.[5][11] He received the Society Award (for "outstanding technical contributions and leadership") from the IEEE Signal Processing Society for 1997,[20] and the ESCA Medal for Scientific Achievement in 1999.[21] He was a recipient of a IEEE Third Millenium Medal in 2000, the ELRA's first (2004) Antonio Zampolli Prize,[22] the 2005 James L. Flanagan Speech and Audio Processing Award,[23] and the 2009 Lifetime Achievement Award from the Association for Computational Linguistics.[11][12] He received a honoris causa Ph.D. from Charles University in 2001,[24] was elected to the National Academy of Engineering in 2006 and made one of twelve inaugural fellows of the International Speech Communication Association in 2008.[5]
| Preceded by Fumitada Itakura |
IEEE Signal Processing Society Award 1997 |
Succeeded by Bernard Widrow |
| Preceded by Mario Rossi |
ISCA Medal for Scientific Achievement 1999 |
Succeeded by Louis Pols |
| Preceded by Gunnar Fant |
IEEE James L. Flanagan Speech and Audio Processing Award 2005 |
Succeeded by James D. Johnston |
| Preceded by Yorick Wilks |
ACL Lifetime Achievement Award 2009 |
Succeeded by William Aaron Woods |